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abstract 



Rating is a process in which we relate observed perfor- 
mance to a designated concept of standard. This process is 
usually carried out to determine the "standard time". Per- 
forming the job in the standard time is called "standard 
performance* . 

There are two major methods of rating, 

(1) methods using a mathematical formula, 

(3) methods using judgement. 

The mathematical formula methods are not considered as 
being accurate and are not in general uee. Industry largely 
makes use of those methods involving judgement on the part 
of the rater. 

Time study engineers in industry today have been on the 
job various lengths of time; have received their time study 
education from various sources, and utilize different con- 
cepts of "standard performance". It is believed that their 
accuracy in rating is affected by these and other factors 
such as area of employment, size of plant, as reflected in 
number of employees, and size of city in which plant is lo- 
cated. The objective of this thesis was to determine if any 
or all of these factors affect the accuracy and consistency 
of the ratings made by a typical group of time study engineer 
from the Mid-West region. This was accomplished by analyzing 
the ratings of job films in accordance with the following 
breakdowns: 
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1. Entire group. 

3. Area. 

3. Experience. 

4. Initial source of time study training. 

5. Concept of "standard performance". 

6. Size of plant. 

7. Size of city in which plant is located. 

*ith the engineers assembled in a room, films of an 
operator performing a synthetic task were projected. The 
job was a simple laboratory operation, representative of 
light factory work. The films were divided into 12 loops 
with the operator working at a different pace in each loop. 
Loops were projected in a random order of pace. Engineers 
were instructed to rate by cycle and to employ any method 
of rating that they desired. Method of performing the job 
was stipulated to be correct. In order to place all ratings 
on a mutually comparable basis and to simplify the mathe- 
matical problem, Rating Conversion Scales were used by each 
man to convert his raw rating to that figure he would have 
assigned had he assumed 130 to be the maximum numerical value 
he would expect of the typical operator in his plant. 

Conclusions. 

An analysis of the results of a group of typical time 
study engineers when rating job films leads to the following 
conclusions: 
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8. Size of city . Size of city in which the plant is 



located apparently has little or no bearing on the raters' 
accuracy. 

9. Consistency followed by the same direction as 
accuracy in all cases. It was, however, slightly higher. 
This was to be expected from the manner in which the data 
were handled. 

This analysis suggests that when rating a job film we 
can probably expect of time study engineers, in terms of the 
percentage of personnel within limits indicated, the follow- 
ing degrees of accuracy: 

±5$ level: more than one third 38$. 

± 7 ^% level: almost one half 48*1. 

+10$ level: almost two thirds 62$. 

±20$ level: the great majority 86$. 
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ah evaluation of time study ratings of a SYNTH ft I C TASK 

INTRODUCTION 

Motion end tine study refers to the analysis cf the 
work and time necessary to do a certain job".^ To find the 
time necessary we usually resort to a 3top watch time study. 

A stop watch time study is accomplished by observing an 
operator performing the task and recording the time obtained. 
However, we note that if we should time another operator per- 
forming identically the same job, in the same manner, we very 
probably would obtain a different time of accomplishment. 

And if we studied a third operator we might get a third 
value of time. The question now arises, “which one of these 
workers is performing the task in the correct amount of time’*? 
The answer to this question is required to establish a 
"standard time" for the job in question. Performing the 
job in the standard time ia called "standard performance". 

One definition of standard time given by Mundel*' i6, 

"the time necessary to do a certain job, defined as to 
method and conditions surrounding the work, by a operator 
physically fit for the job as the typical operator who could 
be expected on the job, possessing sufficient skill to do 



1 M. E. Mundel , Systematic Motion and Time Study , p. 3, 
Prentice-Hall, Inc., New York, 1947. 



3 Ibid. , p. 147. 
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tne job properly, adapted to the job, and working at a pace 
100/130 of the maximum he could maintain for the working day 
without harmful physical effects". But with all of these 
conditions understood and defined, it is still necessary to 
determine what relationship the performance of the operator 
observed bears to the pace defined as standard. The process 
of relating observed performance to a designated standard is 
called "rating", and further discussion will be confined to 
this subject. 

There are two major methods of rating, 

(1) methods using a mathematical formula, 

(2) methods using judgement. 

(a) Multiple factor systems, 

(b) single factor systems. 

Because of the great number of variables affecting the 
operators time the mathematical formula methods are not con- 
sidered as being accurate, are not in general use, and will 
not be considered here. 

An example of the Judgement method is the "effort" 
rating plan. This plan involves two steps, 

(1) judge the difficulty of the job and imagine a 
concept of reasonable performance, 

(3) compares what he sees to this concept. 

To carry out this method the time study man should be fami- 
liar with the visible appearance of the steps in the range 
of effort, as reflected in pace, on each type of job he ob- 
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serves/ In a. plant with a variety of jobs this is not always 

feasible. 

Now judgement must involve the determination of the effect 
of the operator’s skill, aptitude, and effort on his perform- 

4 

ance. The system advanced by Mundel accomplishes the above 
and entails the following steps on the part of the rater, 

(1) compares the workers pace to a fixed concept of 
speed, 

(3) adjusts the observed value for difficulty of the job. 
It becomes obvious that a standard pace must be defined. 
The standard pace may be a man walking on a level at 3 miles 
per hour. Also it could be a person dealing a deck of cards 
into 4 equal stacks in one half of a minute. ^ It is evident 
some value or representation of pace must be made in order 
that judgement of pace be possible and effective. A physical 
embodiment of this designated pace could be a film cf an 
operator walking or dealing cards at the respective rates 
indicated above. Some time study men have their own concept 
of standard performance and some utilize a film as previously 

5 

mentioned. When using the Presgrave system a different con- 
cept of the standard pace for each job is required. 

3 Ibid., p. 157. 

4 Mundel, op. cit., p. 157. 

5 Ralph Presgrave, Dynamics of Time Study . 2nd ed. , 
p. 154, McCraw Hill Book Co., Inc., Hew York, 1945. 
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Rating May be expressed as points per hour or as a 
percentage figure. Percentage numbers are more widely used 
and will be employed in this thesis. There are various 
percentage systems in use but the following will be employed 
here: the maximum performance that could be expected of the 

typical worker, for the entire working day, without harmful 
physical effects will be 130 and standard performance will 
be 100/130 of this value. 

To adjust for difficulty of performance on different 
jobs, tables are prepared with certain allowances for adjus- 
tment purposes. 

Time study engineers in industry today have been on the 
job various lengths of time; have received their time study 
education from various sources, and utilize different con- 
cepts of "standard performance". It is believed that their 
accuracy in rating 1 b affected by these and other factors 
such as area of employment, size of plant, size of city in 
which the plant is located. 



6 Ralph M. Barnes, Motion and Time Study . 3rd ed., 
p. 353, John Wiley A Sons, Inc., New York, 1949. 
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OBJECT 

The object of this thesis is to determine if any or all 
of these aforementioned factors affect the accuracy and con- 
sistency of ratings made by typical group of time study 
engineers from the Mid-S7est region. This object will be 
investigated by analyzing the ratings of job films in 
accordance with the following breakdowns: 

1. Entire group. 

2. Area. 

a. Northern Mid-West, less Michigan. 

b. Central Kid-West. 

c. Southern Mid-West. 

d. Michigan. 

3. Experience. 

a. 0-6 months. 

b. 6 months - 3 years. 

c. 3-4 years. 

d. Over 4 year 3. 

4. Initial source of time study training. 

a. College. 

b. Company. 

5. Concept of "standard performance". 

a. Rater usee ovn concept . 

b. Rater uses a film or some other embodiment. 

6. Size of plant, (number of employees), 
a. 0-101 men. 



b. 101-1000 men. 
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c. Over 1000 men. 



Size of city in which plant is located. 

a. Under 5000. 

b. 5000-10000. 

c. 10000-25000. 

d. 25000-50000. 
c. 50000-100000. 
f. Over 100000. 
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*rocsdure-part A 

In order to carry out the objectives of this thesis it 
was necessary to have a typical group of time study engineers 
rate a task or a number of taBks. To obtain this data In- 
dustrial Engineers from various firms located in Indiana, 
Illinois, Michigan, Ohio, and Wisconsin were invited to attend 
a "work session" at Purdue University on the 15th of *5arch, 
1950. 

Data could have been obtained by having the raters view 
either a fill* of an operator performing a task or view an 
actual operator. For various reasons a film was chosen rather 
than en operator. Some of these reasons are: 

1. A film was already available thus obviating the 
necessity of training an operator. 

2. A film io more convenient as it can be viewed by a 
much larger group than could an operator working at a task. 

3. Method and skill can be maintained more consistently 
and accurately by means of a film since many operetors are 
distracted when performing in the presence of an audience. 

7 

4. It was shown by Margolin that time study engineers 
rate more accurately and consistently from a job film than 
from an operator. 

The film selected consisted of an operator performing a 
simple task, a job description of which is appended. The 

7 Louis Margolin, ”A Comparison of Two Methods of Pre- 
sentation For Time Study Ratings", Unpubliened Masters Thesis, 
Purdue University, 1S48. 
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film as prepared in iS'i9 under the supervision of Lebrer.^ 

It ’ u3 divided into 36 parts in which the operator performed 
at a different pace in each part. The film had been carefully 
edited to eliminate any fumbles, false movements or any other 
irregularities. Considerations involved in the selection of 
this fila were: 

1. It involves sharp clear motions and elements. 

2. Includes motions of the hands and arm3 that arc 
typical of light faotory work. 

Twelve loops were prepared from the film. They were 
carefully checked to insure that the ratio between the elem- 
ents of a given loop in each cycle were the same, hoops were 
arranged in a random order of pace for projection purposes. 

It being realized that different time study men utilize 
dif 1 erent numerical maximum percentage values as expectable 
of the typical worker, rating conversion scales were prepared 
for their use. In this manner they could convert their raw 
rating to the one in which 130 represents the typical maximum. 

Two IBI4 mark sensing cards were prepared for issuance 
to each man. These cards were punched with identifying data 
for each man. It was arranged, that each man would then enter 
his "converted rating" directly on the IBW cards. 



8 R. K. Lehrer, "Development and Evaluation of a Pace 
Scale for Time Study Rating", Unpublished Doctors Thesis, 
Purdue University, 1949. 
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procedure-part b 

The engineers assembled in the lecture room of the 
Electrical Engineering building and after a greeting by Dr. 

M. £. Munciel, Professor of Industrial Engineering, General 
Engineering Department, Purdue University, a questionnaire 
was handed out to each man. This questionnaire was for the 
purpose of collecting information for the thesis of Borrus.^ 
It further served the purpose of procuring the necessary 
information such as area, experience, etc., of each man for 
this thesis. 

On completion of the questionnaire the IBM mark sensing 
cards, special pencils, rating conversion scales, were issued 
to each man. Instructions for their use was given. 

Engineers were requested to rate by cycle, using their 
own system of rating. Method of the Job was stipulated to be 
correct. They were further reminded to use the rating con- 
version scales in order that all ratings would be on a 
mutually comparable basis. 

A loop was projected for a short time to familiarise 
each man with the method of the job. On completion of this 
showing the session began. Loops were projected at the came 
speed at which they were made. About 2 minutes were allowed 
for projection and about 1 minute for recording of ratings. 
This procedure was continued until all 13 paces were shown. 



9 Bernard S. Borrus, "The Present State of Time Study", 
Unpublished Masters thesis, Purdue University, 1950. 
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PROCEDURE- part 0 

After all 13 paces had been rated the IBk! cards were 
delivered to the I3H Computing Laboratory where all necessary 
data were punched on the cards. The IBlf Laboratory supplied 
the following information for each pace and for each categ- 
ory of breakdown: 

1. dumber of raters. 

2. Totals of their scores. 

5. All ratings made. 

4. Humber of raters with a specific score. 

Kith this information, corrected values for each pace w&s 
determined. Uathematical procedure for determining this is con- 
tained in Appendix B. With the corrected value of each pace 
obtained, and the information from IBif, analyz&tion for 
accuracy was made by noting the number and percentage of 
raters xrithin £5$, £7f^, 1.10$. £20$ of the corrected values 
for each pace and for each category of breakdown. 

To analyze for consistency, the same procedure was employ- 
ed except the average of the raters' values was used as the 
measure about which the variation was appraised. 

To arrive at conclusions the data were investigated by 
inspection and by the "analysis-of-variance" technique. The 
analyBis-of-variance technique was employee at the £5$ and 
£.10$ level of accuracy in all cases and at the £30$ in a fer. 
Appendix B has a short discussion on the analyeis-of-variance 
technique. 









hx»0 •‘•rj - J * 1 * x T- C fc«.t Ojpyt tJ X., 

• '*-• V -'to... t Vi! -.., OJ ft. ,«/!** 

! J ' •« .*•*« - i, - - ,,..6 

-r-J o *o. Ml 1, ,5„. d=A, -ot fttlJM.loJai .X,.. 

:iwoi Ho x*o 

Z 1c *t*cr^f* , x 

IfriU -I*JoT .S 

* v -* V auoj Wi IX; 

• OXliOS-Ti to i.cJejj. .4. 

-.* 19*9 7 C •rolav feajorrxoo . ljt „ 5 

*» ‘i i; y i.rl *.i3 im 6 xol »x«ft»ocrq 

tl " ,, "** *>**>***co Bti) ajiT! .8 rifm*,, : A ui 6* u ** 

*"• <»"* wl nc£^»tol.»i »rf; a- 

10 ** ■'**>**'' *«» *»<*•»« *r./*oa t<X »&*,* ^ X0«^00« 

b,*> ,noo •<* Xo V; f r # 0 l ± ,*iVf ,M4 aliu X, 

. wofriiwrrxf Xo Y?o sj d ao?s toi bsu. »s r; .u xol 
-Xol^ *«« wimwoTn ***. 9At ,*><,•*. 1 wo x*‘r « x i_, , 1<f L ? 

* ’* " * W; *" ‘ ouI ** ••»«'** »«s ** M^‘V. X Xsaos 

• L* «X otijqa .... XT 3 i 3 * t'l t ,Y 9(i3 XiOJtiJ* Xu- d<» *T£J--!-.. 

Td i^rf^vn/ *x<^ « ltrj » X, jr 

-•■’ .*r: V k»i ., • P)) , ,*_ r , Jt , 4 , u 

"± iX *., •«. I..-- M» »wfvXnik-s^ .;l,.r^ 

1 u '**'•£ J ' 1 n " ,S3l ° II*‘ Tl *iO>TU^OX ►.. JfOXj4 

; t-r ..... . f, Uxt, ■ t XO Ull M-l JlOd. *.* i * » **.«.,** 

* *. ■ * . r * t 






Raters' averages were plotted against the corrected 
values for each category of breakdown. A line, determined 
by the method of "least squares", was drawn through these 
points. This line indicated the points at which raters 
tended to depart from the and +10& levels of accuracy. 
Appendix B contains a short discussion on the "least squares" 
method of fitting a curve through a set of points. 
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CONCLUSIONS 

>n analysis of the results of a group of typical time 
study engineers, when rating job films of various paces, 
leads to the following conclusions: 

1. For all categories, a plot of the average of the 
ratings against the corrected ratings indicates a trend of 
tne time study engineers to rate too low at the higher values 
and too high at the lower values. 

3. The line best fitted, by mathematical means, to all 
points for the entire group, suggest the trend of this group, 
on the average, to be outside the *5# level at values below 
about 93; the +10$ level at values below about 75; the -5f 
level at values greater than 165; the -10# level at values 
in excess of 220. 

3. Area . The Michigan area had a comparatively low 
degree of acouxacy, the analysie-of-variance test showed it 
to be significant, but since their amount of experience and 
familiarity with the method of rating a job film was not 
determined, and the sample was very small, their score is 
disregarded. Discounting the Michigan category, raters from 
various arena rated essentially the same. Therefore, the 
area in which the plant is located appears to have little 

or no relationship to accuracy. 

4. experience . Groups with 6 months or more experience 
rated about the same but with greater accuracy than those 
with less experience. Those in the 2-4 years category were 
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slightly higher than the rest. The analysis-of-varir.nce teat 
failed to indicate that the low score of the 0-3 months group 
was significant, hut in view of the irregularity cf their 
accuracy, as indicated by inspection of Table 7a, it is 
cug^eated that in general at least 6 months experience is 
recuired to make the qualified time study engineer but that 
accuracy then remains about the same with increased experience. 

5. Education . The accuracy of the raters at the +5% 
level is practically equal. The 1$ difference ir. their 
accuracy is not significant. However, at the +10$ level 
those raters receiving their initial time study training 
from a. company show a significantly, by enalyeis-of-variance 
test, greater accuracy. This suggests, at least for this 
group, at the higher level of accuracy, initial source of 
time study training does not affect accuracy. At a less 
rigid degree of accuracy, time study men receiving initial 
time Gtudy training from the company are more accurate than 
those receiving it from a school. This cay not be true of 
those receiving modern training from either school or 
company. 

6. Concept of Standard Performance . Raters using film 
or some other embodiment of standard performance as common 
practice were superior to those using their own concept 
despite tne fact that no film aid was used in this chase. The 
difference in accuracy was not significant at the ±5 level 

of accuracy but was definitely significant at the 410% level. 
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It is therefore suggested that raters should use a film or 
some oxher embodiment of "standard performance", and this 
reinforces the last statement of conclusion 5. 

7. dumber of Employees . By inspection the raters in 
plants employing less than 101 men rated a little lower than 
those in larger plants. However, this difference is not 
significant. Therefore the sire of plant a& reflected in 
number of plant employees apparently has no effect on raters 
accuracy. 

8. Size of city in whloh plant is located . Inspection 
reveals a wide difference in accuracy between a city of less 
than 5,000 people and a city of 5,000-10,000 people; the 
analys is-of-variance teats reveals this difference to be in- 
significant at the +5$ level of accuracy and barely significant 
at the VIO^ level. It ia therefore suggested that size of city, 
in which the plant is located, does not afiect raters accuracy 
signif icantly. 

9. Consistency followed by the same direction as accuracy 
in all cases. It was, however, slightly higher. This was to 
be expected from the method in which the data were handled. 

10. This analysis suggests that when rating & job film 
we can probably expeot time study engineers in terms of the 
percentage of personnel within limits indicated, to perform 
with the following degrees of accuracy: 

± 5 % level: more than one third 38$. 

level: almost one half 48$. 
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APPENDIX a 
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Breakdown: Tabulation of Numerical Results of all Catcgori 

Computed for Accuracy 
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Table lb 



Breakdown: Tabulation of numerical Results of all Categori 

Computed for Cons latency 
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Table Sa 



Breakdown: Sntire Group. 

Humber of Raters: 73 

Evaluation for accuracy. 

Corrected Sequence ± 5 $ 



Rat ings. 


Shown 


Ho. 
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± 7 ±% ± 10 $ + 20 $ 



Ho. 


$ 


Ho. 


$ 


Ho. 


$ 


46 


64 


48 


67 


66 


22 


32 


44 


47 


65 


67 


93 


46 


64 


57 


79 


66 


92 


44 


61 


50 


69 


63 


87 


45 


65 


56 


78 


67 


93 


45 


65 


53 


73 


65 


90 


33 


44 


42 


58 


63 


87 


40 


55 


49 


68 


64 


89 


33 


44 


42 


58 


57 


79 


38 


39 


35 


49 


60 


83 


31 


29 


28 


39 


53 


74 


36 


36 


32 


44 


54 


75 


35 


48 $ 


45 


63$ 


62 


86 $ 



ei 



-S *1-! 



.qu«jxc -•'lit. j • . I- bsucjsiij 
: I. to tj'j. i;'J 

• V -ruona x.3 1 a*. 



4 


*' -t± 




r^S4 


Jra 


4 


eo/t9t/« vC 


£>•« . av z*/. 


I 


* si 


# 


. O* 


> 


.at 


* 


.0.* 


OVlUf 




*e 


** 




s. 


*3 


a* 


7£ 




PI 


'WI 


$e 


?♦ 


a? 


7* 






r.a 


& 


* 






w 


#* 


V 3 


>V 


6£ 


s *, 


>2 


II 


v:.i 


t% 




ca 


03 


I a 


!•* 


y* 


fc£ 


i 




ft 


v* 


*7 


ae 


<tc 


a > 


u 


IS 


I 






as 


SY 


E3 


aa 


3£ 


v> 




d 


Ixi 


r© 


r,5; 


9 d 


S£ 


**> 


It* 




d 


01 


JOI 


% 


ia 






aa 


0# 


s> 


(“ 


s 


at 


3V 


fc 




s> 




22 


as 


at 


a 




v> 




n 


3« 




3E 


ear 


#.s 


a 


St 


£7 


43 


T, 


6H 


■z 


IS 


»t 


?I 


« 


15 


a? 


*4 




Si 


62 


as 


d. 


Hi 


V 


A 




iO 


'* c 


0* 


Jk*£ 


aa 


*6S 






*v<t 



20 



Table 2b 



Breakdown; Entire Group 
Humber of Raters: 72 

Evaluation for consistency. 



Average 

Ratings 


Sequence 

Snown 


+5$ 

Ko. 


$ 


no. $ 


iio* 

Bo. $ 


420$ 
No. $ 


149 
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50 


44 


61 


52 
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Table 3a 



Breakdown: Area-Northern Mid-Weet, less Michigan. 

Number of Haters: 20 

Evaluation for accuracy. 

Corrected Sequence ±5$ +7l$ +10$ +20$ 



Ratings 


Shown 
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No. 
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No. 


$ 


No. 
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Table 3b 



Breakdown: Area-Northern Mid-West, less Michigan. 

Buasber of Raters: 30 

Evaluation for consistency. 



Average Sequence £5% £10% £30% 

Ratings, Ohown 







No. 


% 


No. 


% 


No. 


% 


No. 


% 
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Table 4a 



Breakdown: Area-Central Kid-rest. 

Number of Raters: 31 

Evaluation for accuracy. 



Corrected 


Sequence 


45$ 


±n< 


±m 


42 


0 $ 


Ratings 


Shown 


No. 


$ 


No. 


$ 


No. 


$ 


Ho. 


$ 


150 
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61 
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65 
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74 


30 
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Table 4b 



Breakdown; Area-Central Mid- 
Number of Raters: 31 

Evaluation for consistency. 



Average 

Ratings 


Cequence 

Shown 


±5$ 
No. $ 
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13 
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Arerage: 
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± 7 # ± 10 $ + 20 $ 
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TaOle 5a 

Breafcdo*n: Area Southern Jr'id-1?est. 



Jfumoer of Waters: 12 

Evaluation for accuracy. 

Corrected Sequence ±9% 



Ratings 
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Table 5b 

Breakdown: Area Southern Mid-lfest. 

Husaoar of Raters; 12. 

Evaluation for consistency. 



Average "equence 45$ £7$$ +1C$ £20$ 

Ratings L'hown — — — 
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Ko. 
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10 


83 
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11 
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92 


144 
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50 
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92 
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Breakdown; Area Michigan 

Eutnber of Rascrs: 9 

Evaluation for accuracy. 



Table 6a 



Corrected 

Ratings 


Sequence 

Shown 


±P;° 
yo, £ 


165 


13 


l 


11 


152 
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11 


145 


11 
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22 


140 


8 
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11 


130 
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Taole 6b 



Breakdown: Area Michigan. 

Number of Raters: 9. 

Evaluation for consistency. 



Average 

Ratings 


Sequence 

Shown 


No . $ 


±?i* 

No. $ 


£10$ 
No. $ 


£20$ 
No. $ 
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Table 7a 



Breakdown: Experience 0-6 

Number of Raters: 6 

Evaluation for accuracy. 

Corrected Sequence ±5$ 



Ratings 


3hown 


No. 


$ 


152 


12 


4 


67 


140 


4 


3 


50 


134 


11 


3 


50 
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33 
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Months. 



47^$ 410$ 420$ 
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Table 7b 

Breakdown: Experience 0-6 Montba. 

Humber of Raters; 6. 

Evaluation for consistency. 



Average 


Sequence 


+5$ 


±7V '' 


+10$ 


+20$ 


Ratings 


Shown 


Ho. 


* 


Eo. 


$ 


Ho. 


$ 


Ho. 


$ 


145 


13 


3 


33 


4 


67 
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100 


6 


100 


133 
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33 
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67 
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6 


100 
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5 


83 


5 


83 


123 


1 


3 


50 


4 


67 


5 


83 


6 


100 


110 


5 


3 


33 


2 


33 


2 


33 


4 


67 


108 


10 


3 


50 


3 


50 


3 


50 


5 


83 


103 


3 


4 


67 


4 


67 


4 


67 


5 


83 


9G 


6 


1 


17 


3 


33 


4 


67 


4 


67 


95 


3 


3 


50 


3 


50 


3 


50 


4 


67 


84 


9 


0 


0 


2 


33 


3 


33 


4 


67 


83 


7 


3 


33 


3 


33 


3 


33 


4 


67 
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Table 8a 



Breakdown: Experience 6 Months to 3 Years. 



Number of Raters: 16 

Evaluation for accuracy. 

Corrected Sequence +0$ 



Ratings 


Shown 


Ko. 


* 


156 


12 


8 


50 


144 


4 
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37 


138 


11 
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31 


133 
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+10 % ±20% 
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No. 
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Table 8b 

Bre&xdown: Experience 6 Months to 3 Years. 

Number of Raters: 16 

Evaluation for consistency. 



Average 


Sequence 




45$ 




410$ 


420* 




Ratings 


Shown. 


No. 


$ 
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No. 


$ 


No . 
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Table 9a 



Breakdown: Experience 3 To 4 Years. 



Kuaber of Raters: 19. 

Evaluation for accuracy. 

Corrected Sequence +5$ 



Rat ingB 


Shewn 


No. 


# 
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47 
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11 
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47 £$ 410$ +20$ 
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Table 9b 

Breakdown: Experience 2 To 4 Yeare. 

Wuaber of Haters: 19. 

Evaluation for conn latency. 



Average 

Ratings 


Sequence 

Shown 


±5% 
No. % 


Ho. i 


±10% 

No. 


±20% 

No. 


% 
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Table 10a 



Breakdown: Experience— Over 4 Years. 

Number of Raters: 29. 

Evaluation for accuracy. 



Corrected Sequence 45^ 



Ratings 


Shown 


No. 


* 
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13 


14 


48 
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Table 10b 



Breakdown: Experience Over 4 Years. 

Number of Haters; 39 
Evaluation for con a latency. 



Average 
Rat ix^ga 


Sequence 

Shown 


15# 

Ko. $ 


tm 

No • f 


±10$ 
Ko. $ 


±30$ 
NO. $ 
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Table 11a 

Breakdown: Education Initial Time Study Training Received 

In College. 

Number of Raters: 22. 

Evaluation for accuracy. 



Corrected 

Ratings 


Sequence 

Shown 


V5$ 
Ho. $ 


Ho. + 


110$ 

No. 


$ 


130$ 

No. 


$ 


156 


12 


13 


59 


15 


68 


17 


77 
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87 
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13 
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30 


91 
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Table lib 

Breakdown: Education Initial Tine Study Training Received 

In College. 

Number of Raters: 33 

Evaluation for consistency. 



Average 


Sequence 


+5$ 


r'i* 


410^ 




420$> 




Ratings 


SJaown 


Ko. 


£ 


No. 


* 


KO. 


i 


Ko. 


* 
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Table 12a 



Breakdown: Education — -Initial Time Study Training Received 

Kith the Company. 

Humber of Raters: 45. 

Evaluation for accuracy. 

Corrected Sequence +5$ 47-^% +10% 430% 



Ratings 


Shown 


S 

• i 
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No. 
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1 P 
ta 
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No. 
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Table 13b 



Breakdown: Education Initial Time Study Training Received 

With the Company. 

Humber of Raters; 45 

Evaluation for consistency. 



Average 

Ratings 


Sequence 

Shown 


45$ 

Bo. $ 


HO. 


410$ 
Wo. i 


430$ 

Bo. $ 
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13 
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Table 13a 

Breakdown: Rater Uses His Own Concept of Btand&rd Performance. 

Nuaber of Raters; 54 
Evaluation for accuracy. 

Corrected Sequence £5$ 47| 4 £10$ £20$ 



Ratings? 
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No. 


* 


Ho. 
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No. 


i 
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Table 13b 



Breakdown: Rater Uses His O tin Concept of Otandard Perforasanc 

number of Jiaterc: 54 

Evaluation for consistency. 



Average Sequence 4Tjr^ 410$ +20 ^ 

Ratings Chown ~ 
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Breakdown; Rater Uses Borne Film or Other Embodiment of 
Standard Performance. 

Number of Raters; 15. 

Evaluation for accuracy. 

Corrected Sequence £5$ 47J|$ £10$ £30$ 



Ratings 


Shown 


No. 


$ 


No. 


$ 


No. 


$ 


Ho. 


$ 
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Table 14b 

Breakdown: Hater Uses Some Film or Otner Emoodiment of 

standard Performance. 

Humber of Raters; 15 
Evaluation for consistency. 



Average 


Sequence 


45 $ 


£ 4 * 


+10$ 


+20$ 


Ratings 
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t 


Ho. 


$ 


Ho. 


$ 


Ho. 
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Table 15 a 

Breakdown: “lant employees number less than 101. 

Number of raters: 10 

TJvaluatlon for accuracy. 

Corrected Sequence +5 4 47M; 410$ 420$. 



Ratings 


CT hown 


No. 


$ 


No. 


4 


No. 
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Table 15b 



Breakdown; Plant employees number lees than 101. 
Humber of raters; 10 
Evaluation for consistency. 



Average 
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Table 16& 



Breakdown: 


Plant 


employ 


ees 


number 


0 

M 

1 


■1000. 








dumber of 


raters: 


37. 
















"valuation 


i for accuracy. 
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Table 16b 



Breakdown: Plant employees number 101-1000. 

Kumoer of raters: 37. 

Evaluation for consistency. 



Average Sequence 45^ 47^$ 4 10$ +30$ 

Ratings Shown — - 



u 




No. 
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No. 
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No. 
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Table 17a 



Breakdown: °lant employee'; number greater then 1000. 

Number of raters: 21 

Evaluation 1 or accuracy. 
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Table 17d 

Breakdown: Plant employees number greater than 1000. 

1 ut-«o«r of raters: SI 

Evaluation for coi.sistency. 
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Table 18a 



Bre'.xdown: 


Size 


of torn 


les 


Number of 


ratero: 


17 




rv&luaticn 


i for accuracy. 
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No. 4 


150 


13 


14 


83 


139 


4 


8 


47 


13c 


11 


13 


71 


128 


8 


4 


34 


119 


1 


13 


77 


114 


5 


10 


59 


102 


10 


3 


13 


9L 


3 


b 


47 


93 


6 


6 


35 


90 


3 


8 


35 


79 


9 


4 


24 


77 


7 


7 


41 


Average: 




8 


47% 



than 5000. 



47lK> +10$ +20$ 



No. 


* 


'To. 


i 


ITo. 


$ 


15 


88 


15 


88 


17 


100 


13 


77 


14 


83 


17 


10Q 


14 


83 


13 


94 


17 


100 


8 


47 


13 


75 


16 


94 


15 


88 


15 


88 


17 


IOC 


13 


75 


15 


88 


16 


94 


5 


29 


5 


29 


16 


94 


10 


59 


15 


88 


17 


100 


6 


35 


6 


35 


16 


94 


13 


71 


13 


75 


16 


94 


3 


35 


6 


35 


13 


71 


10 


59 


11 


65 


14 


83 


11 


62* 


12 


71* 


16 


84* 



'1 flea: 



ic 



- " 0.: »udJ r-c. )o e It- :>r t . -.*5? 

' » : 2 t J ■ t ^ *3 *e/ yl 

.TDfXaOt : 04 3: . ?• c I. • 



* i 


• 0 ± 


ff-i 


i 




* TOJlTOp 


| 


* 0 


« 


.0 


| 


.or 


* 


.of 


JST.n 1 


1 falli 


CVI 


U 


« • 


21 


Of 


at 


S6 


11 


i-’I 


«fx 


' 01 


VI 




a 


V 


Cl 


V* 


0 


• 


Wa 


joa 


Vi 




81 


Ctl 


H 


IV 


tl 


11 


•'.51 


M 


at 


av 


Si 


Vfr 


8 


IS 


0^ 


3 


VSi 




vr 


p? 


31 


88 


Cl 


vv 


SI 


1 


; *1 


-V 


81 


a »i 


ax 


ev 


SI 


6; 


CI 


o 


ril 


M 


8a 


en 


a 


fes 


a 


OJ 


8 


Ua 


Wi 


001 


VI 


a ( , 


at 


Uu 


01 


V- 


8 


l» 


w'_ 


#n 


81 


at 


0 


as 


D 


*S 


a 


6 


66 


111 


81 


av 


SI 


IV 


8i 


at 


a 


S 


Of 


IV 


fa 


a< 


8 


as 


» 


IS 


1 


« 


W 




*£ 


aa 


I- 




01 


I> 


V 


V 


S V 


>00 




vtv 


VI 


*r» 


II 


*V1 


e 




:*v« 'i"v> 



53 



Table 18b 

Breakdown: Size of to^n leas than 5000. 

Nusuber of raters: 17. 

Evaluation for consistency. 



Average 

Ratings 


Sequence 
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Table 19 a 

Breakdown; Size of town 5000 - 10,000. 

of raters: 7. 

^v^lu^tion for accuracy. 
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Table 19b 

r -kdo»n: "Size of town 5,0o0 - 10,000. 

Number of raters: 7. 

Oviluatioi. for oonsiBtency. 
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Table 20a 



Breakdown: Size of town 10,000 - 25,000. 

Number of raters: 19 

*’valuati~n for accuracy. 
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Table 20b 



^Sre-akdo r.: "’Ire of to wn 10,000 - 25,000 

Tiuaoer of raters: 19 

,v>uu'; t ion for consistency. 
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Table 31a 

Bresxdcwn: Size of town 25,0u0 - 50,000. 

umber of Raters; 10. 
valuation for accuracy. 
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Table 31b 

^re^Kdown: 8i*e of town 35,000 - 50,000. 

?ru*ber of raters: 10. 

:,v luation for consistency . 
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Table 23a 

Breakdown; Size of town 50,000 - 100,000. 
’lumber of raters: 7. 

Evaluation for accuracy. 
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Table 22b 



Breakdown; Size of town 50,000 - 100,000 
Humber of raters: 7. 

Evaluation for consistency. 



Average 
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T&blo 833. 

Breakdown: Size of town over 100,000. 

Number of raters: 11. 

Evaluation for accuracy. 

Corrected Sequence V5$ 47£-$ 41 Of' £20$ 
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Table 23b 

Brea*do>m: Size of town over 100,000. 

Husaber of raters: 11. 

Evaluation for consistency. 

Average Sequence £5$ +7£$ £ 10$ £20$ 



Ratings 
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Table 24 

Results of Analy ais-of-Variance Tests 



Breakdown 


f c05 


F clO 


r 05 


Significant 


Area 


3.21 


6.64 


3.95 


Yes 


Experience 


1.11 


1.67 


2.88 


TiO 


Educat ion 
Concept of 


6.48 


7.11 


243-Fc05 

4.84F cl o 


At 10$ 


"standard performance" 3.05 


8.64 


4.84 


At 105* 


lfo. of employees 


1.03 


3.30 


3.44 


No 


Size of city 


3.03 


2.60 


2.38 


Barely at 
10 * 



^c06 F cl0 indic ate computed F* s at the +5# and 410$ 
levels of accuracy. 

F 05 is tabulated values of F at the 5# level of 



significance 
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Fig. 1 




60 70 80 90 100 110 120 130 140 150 160 170 180 



X 



65 



Fig. 2 
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Fig. 3 
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Fig. 4 



Y 



180 

170 

160 

150 

140 

130 

120 

110 

100 

90 

80 

70 

60 




60 70 80 90 100 110 120 130 140 150 160 170 180 X 




68 



Fig. 5 




69 



Fig. 6 
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Fig. 7 
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Fig. 8 
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Fig. 9 
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Fig. 10 




_ I 1 I . I 1 1 . - ■ 

60 70 80 90 100 110 120 130 140 150 160 170 180 X 



74 



i*ie. 11 
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Fig. 12 
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Fig. 13 
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Fig. 14 
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Fig. 15 
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Fig. 16 
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Fig. 17 
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Fig. 18 
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Fig. 19 
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Fig. 20 
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Fig. 21 




85 



71g. 22 
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Fig. 23 

Job description 

left hand. Right hand. 

Transport empty to pins. Hold scribe through 

Grasp one pin. entire cycle. 

Trenoport loaded to scribe. 

Position and transport 
loaded past scribe. 

Transport loaded to pin 
release area. 

Position end release load. 

Job description of the synthetic task: scribe end of 

brass pins, using return chute. 
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mathematical procedure 

Mathematical procedures involved the computing of the 
true pace or “corrected rating" for each pace; the use of the 
"analysis-of -variance" technique, and fitting a line to a 
series of points by the method of "least squares". A short 
discussion on these procedures follow. 

1. Computing the true pace or "corrected rating". 

a. Designate "corrected rating" as X n . 

b. Obtain the average frame count per cycle for 
each pace. Designate the frame count as H n . 

c. To any one of the paces assign the value 100. 

Let this be pace number 10. Designate this a 8 X|^ r 100. 

d. Since each pace bears a fixed relationship to 
all other paces, the relationship being the ratio of their 
respective frame counts, we have the equation x %=C, a 
constant. We now find X^ for each pace from this equation 
and 

e. The "corrected rating", X n , bears the following 
relationship, X n ®KKA, where K ie a constant, a formula 

for deriving X will be given in a subsecuent paragraph. 

f. With the value of K determined, compute X n , 

These values of X n are in a direct ratio with each 
other according to their respective frame count per cycle, 
are related to the raters' average, and represent the near- 
est approach possible to a true rating. 



fHjaaoovi jAor»(wwv7An 



■ . lo ^niJu-jpoo •£*♦ ! '^vlo y ni *o cm I*oli tsscUn'i 

9ii- 10 %-*t e/U iio«t.i 10 1 *'.,;lJ2ii b®/o»iioo* ro * 0*4 ei/U 

cJ riil lil Jt. h»iA ,-j -tXa;osJ *pwiaiT*v-lo-» ^yl***" 
rio.i* a ••' st .uf irtuSl" 1:0 bed*- * * cl/ yd al.t/o: ^iiso 

. ollci *sajL/t'jo-r * ■ ... : p o P i 'uoitl 
. ,jnx i’ll ti© Jost-ioo* no *os: auU eUi jtxlt^r&oC . X 
• n X da "giiJai bafoaiioo* siur i .» 

i.l aloyo laq Idlooc ea *rl auaiavs ut«iac .d 

•ix^ *• JHijOO 9» 'll -do -• "T ‘ ,**Di»q uO*e 

.OJi »wl»/ oo\, xi^i jb »eoij. eul lt» * ti*’ ,ria o* .o 

,C-JX“y|-X *•« -id* ,CX i»r.eu« 10) j e*x *idj Jsi 

ol qid'siioX Xalei b^xi't a »i**d aos do#» .b 

rip 3 1; grfi .K'*d q { . .-r i »uX ,w Or r lio Xl* 

- <0 = a** x * irot3+u>>9 i U .>v* sw ,i Jitii-io • il * 13 ' 
.loilfiuoa «irfr iDOil ■so, xos- ^oTt frX bni < tot •»' .tan xo*) 

. 00 i^ 0 | ba<* 

<>: ipollot *» itf »i ,md tfl » bsloono.j' n * 

48 .ii.noi A JatJ "iM »»i . -,y , i .’ . - * 

. iv #t . J.iau »d w a ti a •vi , *d /XX- ? aatfiw - io\ 
, : ,X SXU'SOO ,(.*> * - sj^f) X C <MlXi>V B jj i3t*’ .1 

,4 0v ' >Jj t r io > ib l ni i' |( X lo 5 -uX. v h.mi: 

,»Xovo Ti»q Jnu:o p* *1 9 *i So^qa**! ii* ii r*J 4 dii' 4 .‘ 0 * 

-i - ot * JO" imtiT'iS'i fi{ r: -dJ 1 oj {•p/^i** r t .'T* 

i *jJ 1 J »* •»>* — i Jwo 



89 



CO 

,? H Ifn 

Proof that Kj This proof was derived by 

^ X£2 

Dr. M. E. Mundel ^'Professor of Industrial Engineering, 
General Engineering Department, Purdue University. 




Problem: 
Given: 
each pace. 



To find the best set of X values about Y = 
X^ , Xg, , Xq, the corrected ratings for 

C 



X. 



X* 

*1* 


X 1 

3 


• 

• 

*0 

fi 

• 


v» 

n 


*1. 


Yg, . .. 


• * * » Y n» 


the average of the raters* 



estimate for each pace. 

K 1b a constant to be selected euch that ?d' J 
equals a minimum, where d is the variation about the line 



Y? X. 



Proof: 2d = (Y x - KXJ) + (Yg - KX*,) 4. ..-4 ( Y n -KX^) . 

«£d 3 = yf - 3KX£Y 1 4 K 2 Xj[ f ... + (Y n - KX^.) 2 . 
The value of K which will permit ^d^ to be a minimum is 
obtained by setting the partial derivative of £d 3 with 
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respect to K equal to 0 and solving for K. 

--SK'Y! + 2KXi - - 2X' Y * 3KX‘ 2 r 0. 

gK 1 1 1 n n n 

Solving for K we have, 

X'Y X + x^y 3 4 ... 4 x^T n 



T" + H 



V 



4 X£ 



73"' 



0(7 



K- 






2 n 

ns i 

2. Analysis-of-variance technique. 

Details of the analysie-of-v&r iance tests are not pre- 
sented here^ but it can be shown that if we have the variable 
X arranged in the following manner, 




and if we assume these values of X are random sample 
values from a normal population with mean ta and variation 0*, ^ 



10 Paul G. Ho el, Introduction to Mathematical Statistics , 
p. 154-161, John Wiley & Sono, Inc. , Hew York, 1949 . 
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a 2<X , - X) 3 (a-1) (b-1) 

~%j[^ “ x i. - x : 3 * x > TSZ) 

C*ai o-i 

The numerator, with (b-1) degrees of freedom, measures 
variation between columns. The denominator, with (a-1) (b-1) 
degrees of freedom, measures the "experimental error", that 
is, variation in the fundamental v&riacle after row and column 
effects have been eliminated. 

In this thesis, a is the number of paces, b is the number 
of breakdowns, X is the percentage of raters within +5$ of the 
corrected rating or any designated level of accuracy. 

F 0 was computed at the +5# and ±10$ levels of accuracy 
for eaoh category of breakdown and w&a compared with the 
tabular value of F at the 5$ level of significance. 

3. Fitting a curve to a set of points by the method of 
"least squares". 

Details of the method of "least squares" will not be 
presented here but involves the use of the following equations. 

aN + o£x. 

£XY 5 a<£X + m£X 2 . 

amintercept, mwslope. 

tfith the values of a and m determined we substitute them 
in the general equation for a straight line, T:«X ■+ a, and 
obtain the line best fitting the available points. 



11 Ibid., p. 79. 
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